Reliability Society Newsletter - August 2012 Feature Article:The Elusive Definition of Requirements Completeness
IEEE Standard 830 "defines the following desirable qualities of a system or software requirement specification (SRS)
Of these qualities, completeness is perhaps the most difficult to achieve [2]. While it is well understood that an incomplete SRS can lead to disastrous effects [2] [3][4], the definition of "completeness" itself is inconsistent throughout the literature. The purpose of this article is to discuss the various definitions of "completeness" in order to help engineers to better communicate their intent and measure this elusive quality. The references also provide a starting point for those interested in further studying the notion of requirements completeness. 1. Requirements Completeness Defined Boehm notes that a requirement is complete "to the extent that all of its parts are present and each part is fully developed" [4]. This definition is corroborated in [3]: "[ completeness is] the sense of whether the specification contains all requirements and whether all requirements contained in the specification are completely specified". Some authors describe " feasible completeness" [5], which is achieved when "there is no relevant statement about the domain, not yet included in the model, such that the additional benefit to the conceptual model from including the relevant statement exceeds the drawbacks of including it." Other authors discuss completeness "with respect to a reference model" [6]. That is, requirements are complete when "all requirement model elements satisfy a match" in a specific interaction pattern library. But these definitions imply the existence of such models or libraries, which in some domains may not exist, presenting a challenge to requirements completeness. Still others introduce "internal completeness" [7] [8], which essentially means that if a concept is mentioned in a requirements specification, it must be elaborated upon within the same document. Multiple other works by Jaffe, Leveson and Hiemdahl follow a common definition stating that requirements are complete when "a response is specified with respect to every possible input and input sequence" [7][9][10]. In [10], the authors also point out that completeness is satisfied only when "every [has] a deterministic behavior defined for every possible input event." This definition is offered within the context of safety crucial real-time systems, however. Other definitions use a different perspective entirely. For example some authors look at the problem from an end-result perspective to state that completeness is "a lack of ambiguity from the implementation perspective. The specification is incomplete if the system behavior is not specified precisely because the required behavior for some events or conditions is omitted or is subject to more than one interpretation" [11]. This definition seems to imply, however, that the ends justify the means. Some authors tackle completeness from the perspective of goal satisfaction, that is, requirements are complete if all stated goals are satisfied [12]. The authors of [13] extend this definition by adding the caveat that "completeness is a notion relative to what is known about the domain." Similarly, in [14] an attempt is made to find incomplete requirements by comparing the items in the requirements specification with a particular ontology. This observation reinforces the notion that the domain, or what is available in an ontology, is not nearly sufficient to ensure completeness and may provide a false sense of security, therefore making completeness validation more difficult. Similar to this approach of utilizing the product of the requirements to help define whether the requirements were complete, is the definition of completeness as the "property that requirements are sufficient to distinguish the desired behavior of the program from that of any other undesired program that might be designed" [15]. This kind of definition can further increase understanding completeness, but provides little assistance in testing for completeness. Some authors have rather limited definitions of completeness due to the particular domain addressed, without specifying a particular nomenclature for the definition. For example, the authors of [16] define completeness as simply the "presence of all event handlers or actions for condition guards of all events". Unfortunately, some studies discuss requirements and their completeness without further defining exactly what completeness means in said domain [17][18][19][20]. In [20], for example, it appears as if the authors utilized a ruleset within the tool studied to verify for completeness, but unfortunately neglected to include that ruleset. Others discuss "reasonable completeness", with no further definition [21]. Conclusions and Future Work It is difficult enough to achieve requirements completeness, especially when the notion of completeness itself is inconsistent. The definition proposed by Boehm [4] seems reasonable, but it is important to interpret completeness within each application domain and in the context of the operational environment for the system in question. It would be helpful if Boehm's or another definition would emerge as a universal standard. But the problem of achieving requirements completeness, whatever that means, is still a very challenging one. Future articles in this newsletter will discuss various ways of increasing requirements completeness. ... References
|
||||||||||||||||||||||||||||||||||||||||||||||||